Repository navigation
API: Define RewrapEncryptionKeys action interface - #18440
Open
dramaticlly wants to merge 2 commits into
Open
dramaticlly wants to merge 2 commits into
dramaticlly wants to merge 2 commits into
Conversation
3 tasks
Re-wrapping every key in a table means rewriting every manifest, which is cost prohibitive for large iceberg tables. The level names the lowest key to regenerate: the keys above it are regenerated too, the keys directly below it are re-wrapped under the new ones, and the keys further below are left unchanged. Generated-by: Claude Code
Contributor
Author
|
@ggershinsky do you think if it's ok to start with KEY_ENCRYPTION_KEY and MANIFEST_KEY depth level? We can always add data file key later in the enum if there's need for the expensive key replacement. |
Contributor
|
sure, makes sense |
Contributor
|
there should also be a MANIFEST_LIST_KEY level, between these two? |
Contributor
|
Thinking of this, we might want to refocus / rename the action, from
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First step of #18240 to define action interface to allow rewrap of encrypted iceberg table.
Added configurable depth level to control how deep the encryption key of iceberg metadata shall be regenerated.
Thought about the option to introduce
rewrapKeysflag on the existing RewriteManifests action and procedure and it was discarded to avoid combining security operation inside a re-clustering/optimizing call, also rewrite manifests can only focused on single partition spec.